Encryption, security, and video optimization

ABSTRACT

Data encryption and Human Pose Estimation based on imaging a body segment. A key for encrypting a data file is generated based on image data that represent a unique biometric feature of a body segment of a user or motion of the user. An image engine executes artificial intelligence to identify matching image data for decrypting the data file. The image engine is further trained to predict changes in image data due to aging, stress, and the like. An avatar associated with the user, which is generated based on a movement pattern of the user, is configurable for generating an encryption key and for use in an avatar-based language.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from U.S. Provisional PatentApplication No. 63/223,783, filed Jul. 20, 2021, the entire disclosureof which is incorporated by reference for all purposes.

BACKGROUND

Encryption is primarily used to protect confidential information. It canbe used as part of a password protection system used to limit access tophysical systems, software, or to proof identity. Many safety/securityencryption systems use simple codes based on alphabet or numerical codesto protect information. For example, a person's social security number,which is a simple numerical series, is a key to many data files but canbe easily acquired with risk of data breach or theft.

Known encryption methods also use mathematical transformations that arenot easily understood or converted back without a key. Most encryptionis a reversible transformation, where the reversal is known asdecryption. Each encryption and decryption function requires acryptographic key, which is typically a string of binary digits. Inorder for the encryption function to transform information intoencrypted information and the decryption function to reverse theencrypted information the encryption and decryption functions must usethe same key (symmetric key). Encryption is used in a wide variety ofapplication, including but not limited to web applications such asSecure Socket Layer (SSL) and Transport Layer Security (TLS),Secure/Multipurpose Internet Mail Extensions (S/MIME), Internet ProtocolSecurity (IPsec).

These types of protections can be stolen, copied, acquired through datahacks or breaches as they are known and stored by many companies (Apple,Google, Facebook, . . . ) as well as smaller technology operators thatregularly request this information and then secretly keep files onindividuals.

Those skilled in the art are familiar with the use of retinal scans,finger prints, and facial recognition for encryption, lockingmechanisms, or security. Even high security techniques such as theseuses static images, which are fairly commonly known and can be stolenrelatively easily. For example, if one knows that a retinal scan isbeing used, one can copy that retinal image.

SUMMARY

Aspects of the present disclosure provide improved datasecurity/protection by examining 3D images or patterns or motion 3Dpatterns of a user rather than simply static surface such as facialrecognition. Depth—microns to millimeters into an object or bodypart—adds complexity to an algorithm for creating a cryptographic keyapplicable to any portion of the encryption and decryption process. Forsecure applications the key can be up to, for example, 192 bits. The keygenerated in accordance with aspects of the present disclosure can bestored by the user or can be stored by an application that uses apassword or facial recognition or other technique to access it. Any orall concepts in this application that apply to encryption and keygeneration can also be used for password generation, controlling accessto a device or file via passwords, and locking/unlocking physicaldevices or structures.

Moreover, these concepts can be added to any existing code, key,security, and/or encryption to add levels or layers for protection. Forexample, adding encryption based on a 3D image, pattern, or motion to anexisting social security or numerical code overcomes the need forregularly changing the security system. In another example, if the userforgets a code or physical key (e.g., for a safety deposit box), a 3Dimage, pattern, or motion of a body or body part that “travels” with theindividual provides an added layer of security.

Keys generated in accordance with aspects of the present disclosure alsoadd the ability to personally encode security, codes, encryptions, etc.to any data package or data file as otherwise expensive third partieshave to create these codes, which again causes risk for a securitybreach. In medicine, HIPAA privacy protection, for example, is improvedas more and more patent information is passed via remote patientmonitoring and remote patient care.

In an aspect, a method for protecting data comprises acquiring initialimage data from a user at a first time. This initial image datarepresents a unique biometric feature of the user. The method alsoincludes generating, based on the initial image data, a key associatedwith the unique biometric feature, encrypting a data file using the key,and acquiring subsequent image data from the user at a second time laterthan the first time. The method further comprises executing an imageengine configured to determine whether the subsequent image data matchesthe initial image data. In this instance, the image engine is trained tocreate a confidence level for matching the initial image data with thesubsequent image data. In response to the confidence level of the imageengine indicating the subsequent image data matches the initial imagedata within a predetermined threshold, the method proceeds to unlockingthe encrypted data file.

In another aspect, a method of creating an avatar-based languageincludes imaging a user and acquiring image data representative thereofand translating the image data for the user into an avatar that isrepresentative of an expression of the user. The method furthercomprises storing the avatar in a centralized database and forming ablockchain to define and render the avatar in the centralized database.

In yet another aspect, a method of generating an avatar associated witha user comprises acquiring image data, which represents uniquemovements, from a user and identifying a movement pattern based on theunique movements of the user. The method also includes executing animage engine to generate a Human Pose Estimation based on the movementpattern and generating the avatar associated with the user based on theHuman Pose Estimation.

Other objects and features of the present disclosure will be in partapparent and in part pointed out herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for encrypting/decrypting a datafile according to an embodiment.

FIG. 2 illustrates acquiring image data for use in the system of FIG. 1.

FIG. 3 is a flow diagram of an example process for locking/unlockingdata according to an embodiment.

FIG. 4 illustrates encryption/decryption using a key according to anembodiment.

FIG. 5 illustrates an example of a user's changing appearance predictedby artificial intelligence according to an embodiment.

FIG. 6 illustrates capturing ECG data from a wearable for use ingenerating an encryption key according to an embodiment.

FIG. 7 a multi-camera arrangement for use in generating a Human PoseEstimation according to an embodiment.

FIG. 8 is a block diagram illustrating a system for authorizing a userbased on the encryption key according to an embodiment.

FIG. 9 illustrates two emoji-type avatars with movement according to anembodiment.

FIG. 10 illustrates virtual reality goggles having an internal cameraaccording to an embodiment.

Corresponding reference numbers indicate corresponding parts throughoutthe drawings.

DETAILED DESCRIPTION

Aspects of the present disclosure provide improved protection and/orsecurity of data. These aspects relate to one or more of: 1) encryption,2) locking mechanisms or security mechanisms to, for example, unlock auser's phone, car, computer, etc., 3) privacy issues to protect data anddata transmissions, and avoid data mining, 4) addresses to be able tovary a user's email addresses, which also helps with either encryptionor also help with privacy issues to prevent data mining of the user'sinterests, 5) a video language/alphabet, 6) distributed computing, and7) blockchain or financial data to generate data or protect dataespecially through these financial sectors where people are using mobilecurrency type concepts.

Machine Vision and Recognition on 3-Dimensional Body Segments:Technology, whether used for unlocking of a cellphone based on facialrecognition or transmission of secured data, requires an encryption key.This key is derivable from a retinal scan, facial recognition,fingerprints, etc., and allows a user to open a phone, for example.According to aspects of the present disclosure, encryption technologybased on other parts of the body, minute facial and skin features, andbody movements rather than simply facial recognition or retinal scanprovide improved security for locking/unlocking, sending secured data,protecting information, and the like.

As shown in FIG. 1 , a system 100 executes a method for protecting data.A camera 102 acquires initial image data at 104 from a user representinga unique biometric feature of the user. For example, the surface of thebody has unique qualities, for example, wrinkles under the eyes, numberof hair count per certain areas, vascular patterns, or skinirregularities. In an embodiment, any of these features captured by theinitial image data are capable of being used to generate an encryptionkey code at 106 associated with the unique biometric feature andspecific to an individual user. Rather than using the entire face, onecan magnify, especially with three-dimensional cameras on phones, tolook at a specific area of the body or face, or to hone down to thewrinkles around the eyes, or to hone into the hair (i.e., around theface or ears, back of the hand, elbow, or knee) or nails. For example,camera 102 acquires image data for use in generating the key from a onecentimeter area in or around the cheek, elbow, or knee looking not justat 3D skin creases but also looking at depth, vascular patterns, hairpatterns, etc. FIG. 2 illustrates close-up image acquisition accordingto an embodiment.

Referring further to FIG. 1 , the key and associated initial image dataare stored at 108. The key is used to encrypting a data file at 110,thus yielding an encrypted data file 112. To later decrypt the encrypteddata file 112, camera 102 acquires subsequent image data at 116 from theuser. The system 100 comprises an image engine 118 for determiningwhether the subsequent image data matches the initial image data. In anembodiment, the image engine 118 is trained to create a confidence levelfor matching the initial image data with the subsequent image data. Inthe event the image engine determines the initial and subsequent imagedata indicate a match within a predetermined threshold confidence level,the key is retrieved at 120 for use in decrypting the encrypted datafile 112 at 122. In this manner, system 100 unlocks a decrypted datafile 124. On the other hand, if the initial and subsequent image data donot match within the confidence level threshold, the encrypted data file112 remains locked.

Aspects of the present disclosure permit creation of a cryptographic keyapplied to any portion of the encryption and decryption process. Forsecure applications the key can be up to, for example, 192 bits. The keygenerated in accordance with aspects of the present disclosure can bestored by the user or can be stored by an application that uses apassword or facial recognition or other technique to access it. Any orall concepts in this application that apply to encryption and keygeneration can also be used for password generation and controllingaccess to a device or file via passwords.

Mobile devices, including smart phones, tablets, smart watches, smartrings, and similar devices that are constantly with individuals can beused to identify, store, and record, data, upload data to the Cloud, andthe like. Such devices can also limit access to the Cloud and storewithin the device itself for security and/or security reasons allowingApps or options. For example, Apple currently loads everything up to theCloud automatically. This can be short circuited in an App or program sothat if one wants to keep this encrypted data within the device itselfor these security mechanisms on the device itself it can block it fromstoring from automatically uploading to the Cloud. This is done morefrequently in Android-based technology, which is not automaticallyuploaded to the Cloud; but this can be controlled by the individual forsecurity and/or management of data.

FIG. 3 illustrates process of FIG. 1 in the form of a flow diagram.

FIG. 4 illustrates further aspects of the encryption/decryption processin accordance with the present disclosure. While described in terms oflocking/unlocking or encrypting/decrypting, it is to be understood thatsystem 100 is configurable for other security environments. For example,the key generated by system 100 is configurable for validatingelectronic signatures, electronically locking and unlocking safetydeposit boxes, and/or authorizing checks, bank withdrawals, credit cardpurchases, and the like. Aspects of the present disclosure are usefulfor anything in need of secure user authorization.

With respect to unlocking as described above, unlocking comprisesunlocking a smartphone, unlocking a computing device, decrypting anencrypted data file, unlocking a vehicle (e.g., automobile, boat,airplane), and/or authorizing a transaction. For instance, a camera onthe vehicle identifies an authorized user using patterns such as gait,specific motion pattern, voice, body part, etc. or combinations of thesefeatures to unlock the vehicle or even start the vehicle as theauthorized user approaches rather than relying or a physical key or fob.A secondary lock may be used to permit placing the vehicle transmissioninto drive. Similarly, security locking/unlocking can be used to permitaccess to data, operate a robotic system, open an app, and the like.

It is to be understood that image data as referred to herein includesnot only static images but also moving images, such as video. Inaddition, image data includes ultrasound, thermography to look attemperature/fluid parameters, Doppler, etc. As described above,conventional biometric recognition techniques base the encryption key onstatic images of the user's face, retina, or fingerprint. In anembodiment, the system of FIG. 1 uses specific sections of the bodyand/or captures video data associated with the user. Where the acquiredimage data comprises video, enhanced security is provided.

Infrared or ultrasound scanning certain sections either in 2D or 3D isavailable through recent technologies that are being embedded into smartphones or added on through mobile phones, smartwatches, etc. Forinstance, close up video captured by a 3D camera permits deeperrecognition of the skin surface, skin creases, vascular patterns, ordepth of the dermis. Moreover, scanning with different illumination suchas ultraviolet, infrared, and other specific wavelengths reveals otherunique characteristics, such as vascular patterns or vascular networksin specific sections of tissue. Different cameras of the same device canlayer different information or detect different information in 3D tolayer encryption or security. In such an embodiment, a first cameradetects a first layer of information, a second camera detects a secondlayers, and so forth. Similarly, a video layer can be added to a staticlayer added to an IR layer and so forth. These techniques are suitablefor use in generating a key based on either specific three-dimensionallocations of body tissue. This combination of patterns can be static orcan be moved to different locations. U.S. Patent Application PublicationNo. 2021/0353785, the entire disclosure of which is incorporated hereinby reference, discloses a scanning algorithm overlay to show areas thathave been treated and those areas that have not been treated.

In an embodiment, the encryption key is generated as a function of acombination of biometric features. Recent scanning technology permitscreating the encryption key through arbitrary patterns or known patternsbased on the user's body as the template and at specific locations. Forexample, the process starts by scanning a whole section of the body,i.e., the face, trunk, legs, or the entire body. Then, the processsequentially looks at patterns in one location or multiple locations ina more specific area to look at patterns and surfaces three-dimensionalvascular, etc. scanning these with a mobile phone. These can be usedthen as a method for encryption, coding, key, etc. If initially lookingat facial recognition, adding 2D or 3D image data of a section of theuser's body (e.g., the ear, neck, etc.) provides improved security. Onelooks at these not just superficially but also skin creases in, forexample, 3D vascular patterns. One looks at these in combination and isable to create new encryption keys, which someone can carry with themlong term. For example, if the user misplaces or loses a computer systemor mobile device, he or she can come back a year later and scan theappropriate locations to unlock device.

While scanning currently exists for bar codes or surfaces (e.g., facialrecognition), handheld scanners in accordance with the presentdisclosure are configurable for performing a 3D scan for codes ortransforming the scan to a mobile code to document items in inventory orbilling or to create a moving code (e.g., video code).

Rather than just having a simple signature to double check, a videocomponent can be added, including vascular patterns, skin patterns,dermal patterns, body part, or even location, environment, or spacewhere one is signing this, to verify the identity of the person signing.2D or even 3D pictures such as signatures, facial recognition, retinalscan, etc. again can be copied and are known and often stored in theCloud so people can access them and potentially steal them. However, ifsomething is specific to one individual's body or the environment whereone is located, the combination of features can be used to furtherencrypt

The specific body location from which the unique biometric informationis acquired can change from time to time. In an embodiment, the locationis moved around sequentially, as well to even further encrypt because ofhow the portion of the individual's body responds to movement. This isdifferent from the state of the art technology because companies alreadyhave relatively broad facial data but this technology focuses on asmaller/deeper portion of an individual's body. Honing in on small bodysegments, such as 1-2 centimeter locations, with specific lights (e.g.,infrared, UV, LIDAR, ultrasound, or varying light patterns) to look atthe vascular flow patterns provides improved security. Similarly,fluorescence can be used to follow the blood flow and/or venous pattern,arterial pattern, or skin edema of the user, which can also change overtime. Subjecting the user to multi-factor authentication (e.g., scanningother bodily sections to confirm the user's identity) before updatingthe user's changing body image data provides improved security. Ratherthan a retinal scan, which may be fixed for a period of time, surfaceirregularities and the like, for example, may change weekly or monthly.These can be linked to other technologies to allow one to protect data.Again, body segments can vary with light, location, surface topography,and surface irregularities, which can be picked up now with most cellphones having three-dimensional cameras. It is also considered that thedetection of different attributes in different locations can be used tocreate a two factor authentication for access or encryption keys. Thisis expandable to multifactor authentication to compensate for aging,scars, where a certain threshold of authentication criteria must be metto unlock the device or give access to the file.

Referring further to FIGS. 1-4 , some unique features of the user maychange over time as the person ages (e.g., melanin levels, pigment,moles, added wrinkles, etc.). All of these can create personalizedencryption or personalized security issues. In an embodiment, imageengine 118 executes artificial intelligence (AI) to assess and/orpredict changes over time, such as aging, changes in daily composition,changes in diurnal composition or during nighttime when one swells lessthan during the daytime when one is more active and tissue shrinks andmobilizes. Aging also causes changes in vascular patterns, skin creasepatterns, dermal thickness patterns, for example. Advantageously, imageengine 118 executes AI algorithms to compensate for this aging so thatone can unlock or change despite these changes in body surfaces or bodytopography over time. In order to save some time for the user the AI canpredict some possible changes so that the user does not have to updatehis/her avatar or go through multi factor authentication.

FIG. 5 illustrates an example of a user's changing appearance over time,namely, growing facial hair. In an embodiment, the AI is a convolutionalneural network that has been trained on data sets from the generalpopulation to estimate aging, hair growth, beard growth, etc. A personaldata set created by capturing images during each login over timeprovides additional information for training the AI. For example, imageengine 118 executes AI to assess and/or predict how a hair pattern wouldchange with aging, hair growth, or motion patterns and then encryptsthis information. Similarly, image engine 118 executes AI to assessand/or predict what changes are with stress. For example, how skin orbody approaches with activities change via stress or with epinephrineinduction. This is either brought into the system or the AI is mapped tosee how these would change with activities. The AI can also be used topredict growth, i.e., a beard, aging, hair patterns, or vascularpatterns for vasodilatation, which can also be used for encryption. Inother words, over a week or a month, body tissue and body surfaces, suchas hair follicles, change as hair grows or skin creases. In the exampleof FIG. 5 , image engine 118 implements AI to predict the user'sappearance at a point in the future.

AI patterns also predict if one changes appearance through plasticsurgery or Botox injections. Despite the aging process, these certainmodifications can be adapted using the AI to understand and encryptdespite the modifications, and to predict how these changes would changethemselves and manage this information. The image engine 118 also canprocess the image data to take measurements on hair per cubiccentimeter, length, curl, angle, movement, and movement with static orelectric changes. For example, when one has static electricity, hairchanges. Such changes can be used to create an encryption pattern byapplying different types of electrical, magnetic, or motion patterns.

Skin creases, wrinkles, 3D vascular patterns, etc., certain treatmentsand medications such as collagen injections and Botox, certain retinolcreams, and the like may alter the skin creases/wrinkles. Potentially,medications may alter the vascular pattern in certain three-dimensionalbody segments that are the subject of scanning for this encryption. AIalgorithms have been used for example to predict aging. If someone takesa face, they can look at an algorithm and they can suggest what thatface would look like in ten, twenty, or thirty years as one ages. Thesesame type of patterns can be used to break if someone has had collageninjections, retinol, and Botox, for example, and how this would affectwrinkling patterns as a secondary check for validation of thisencryption technology.

One embodiment can be configured for pseudorandom patterns forencryption so a user is not simply using facial recognition or retinalscans, but one can truly create a key where one day it may randomly gofrom an eyebrow to a section of the ear to a lower section of the neckand so on. The initial scan can be of the entire face, then future scansto lock and unlock can be a subset of the entire image. This can also beused to generate encryption keys. One can also then change the lightsources from regular ambient light to strobe lighting to different lightfrequencies or wavelengths to capture different attributes. Anotherembodiment can be setup with a detailed video rather than a picture.Since videos with cameras now on mobile devices are very precise andaccurate, one can do the detailed pictures with varying wave lengths oflight. One can, as noted above, add different types of pigment,fluorescents, or coloration to the face either through lighting, makeup,or possibly ingestible material that might show the blood vesselslocations. Infrared or ultraviolet can be used to look at surface and/orsubsurface features. For instance, infrared can be used to tag bloodvessels size, location, vascularity/vascular flow, thermal movement,and/or venous or arterial flow patterns. In the future, mobile devicesmay also have ultrasound, which can be used to scan deep tissue or imagedeeper body parts such as bone tissue, etc., and link these to otherknown encryption based technologies. For example, linking skin patterns,crease patterns, hair patterns, or vascular patterns with or withoutchemical patterns of an individual's body create multipart encryptionprotocols. One can also use, for example, pulse oximeter for looking atthe oxygen content and vessel location. One can use light detection andranging (LiDAR) that is being built into cell phones, for example, inaddition to surface topography, video, and variations in light or motionpatterns. This can be used in conjunctions with wearables such as theApple watch to access the biometric sensors such as pulse, pulse ox,EKG, temperature. FIG. 6 illustrates capturing ECG data by a smartphonevia a Bluetooth or other near-field link from a wearable, such as anApple watch, for use in generating an encryption key.

There are new technologies that are being added on to motion or wearabledevices that include enhanced three-dimensional cameras, opticalcoherence tomography (OCT), infrared, ultraviolet, or ultrasound.Wearable technology can be used to scan tissue body, environment, etc.to help with encryption communication, locking/unlocking technology.These also can be added potentially to medical data, transcription,medical research, recovery, etc.

Gait Patterns with Human-Pose Estimation(HPE): Gait patterns plus HumanPose Estimation can also be used to create encryption using the uniquegait of a user so one can estimate a motion pattern or gait pattern. Inan embodiment, a multi-camera arrangement, such as illustrated in FIG. 7, provides video data for determining the user's gait. To encrypt a gaitpattern, acquired data represents 3D imaging and speed of movement (suchas the movement of the legs and arms) as well as posture. In analternative embodiment, a smart phone or wearable including anaccelerometer provides acceleration data as the user moves in the x, y,and z directions. The output of the accelerometers (or from the HumanPose Estimation) can be converted from a 3D matrix to an encryption key.

One can combine different types of features: gait plus a section of theface on one day; a video of how neck motion occurs on another day; and avascular pattern after one eats or exercises on yet another day. Thesevarious features and movements are detected in an embodiment by addingzone sensors or using wearable smart devices. One can also add electriccharge, temperature, blood pressure, adding enzymes, sweat, chemicalanalysis, or linking tissues with skin surfaces with deeper vascularflow. For example, with any type of chemical or vascular analysis,ultrasound, or other known technologies which can be added to futuremobile device systems. Mobile/wearable devices orintravenous/transdermal agent delivery systems can further enhance thedeep tissue recognition which can be scanned with UV laser, ultrasound,or other known technologies to create unique patterns. Mobile orwearable devices can also be linked to the recurrence of a commonfeature. Energy patterns can also be used, such as electrical, magnetic,wind, or other known motion patterns or patterns that can simulatemotion. For medical applications, differences gait patterns whilecompleting an activity (such as walking or jumping jacks) can be used todetermine what injuries the user might be facing.

Robotic or navigated systems which are directed to going from onesection of the body or another can also be used for encryption, asopposed to randomly going from one section of the body to another. Therobotic or navigated system can go from one body part to the next,scanning certain parts, accepting certain parts, and rejecting certainparts based on a premade protocol. For example, one may want to scan aone (1) centimeter section of the lop and then go to a one (1)centimeter section of the dorsum of the foot. These are automated,scanned, and noted preoperatively especially for encryption technology.One can scan deeper using infrared to see a vascular pattern/deep tissuepattern and combine this with chemical specific patterns for the body.For example, certain people have certain chemical recognition issuessuch as their glucose level, sodium level, or potassium level. Thisinformation can be compared to a digital twin for keys and/or encryptionor it can be more encompassing for general use. The creation of thedigital twin can require a complex multimodal scan of the user(ultrasound, MRI, cat scan, etc.). The digital twin can be used forrecognition, but also for medical diagnostics or planning. The digitaltwin can also be created from the aggregation of pictures and scans usedfor unlocking phones and or files.

This technology can further be linked to voice, blood pressure, orlocations. For example, if someone visualizes a certain portion of thebody and a certain portion of a location at which they are located. Theuser can utilize either an external location combined with an internallocation, as well.

In an embodiment, Human-Pose Estimation and encryption allow function ofa robotic system and efficiencies in use of robotics whether it is inthe Operating Room or for manufacturing applications to standardizebetween one individual and another. Looking at this for axial skeletonso one can use artificial and mixed virtual reality, mixed reality,augmented reality to allow specific activity patterns to be improved ormade more efficient or require less energy.

This navigated system can go from one body part to another accepting orrejecting certain portions of premade protocols. If one is looking atthis in the Operating Room, one can use this to educate or change theperson's patterns to make them more efficient, more optimal, lessstressed, and more function. For example, if one has dyslexia, one canpower this to change dyslexia from an educational perspective via manyof the concepts of encryption technology and can be used to educatepatients with learning disorders. It can also be used in the samefashion to enhance function, work-related activity, under roboticsystems, under exoskeletal, or under Augmented Reality/Virtual Reality.

This also may have substantial value, for example, in any type ofself-driving vehicle. For example, one of the main issues with Teslaself-driving vehicles is that someone can hack into the system. This canbe great recognition for more complex modality such as self-drivingvehicles with complex computer systems, algorithms for encryption, orfor something as simple as cell phone. One can do a scan backwards onthe face and it randomly takes specific sections of a body part, eitherwith still pictures or motion pictures, and then randomly look atspecific locations, specific wavelengths, specific thermal patterns,etc. to determine the encryption and/or approvals. One can add standardencryption such as specific passwords or passcodes using emojis oravatars that are customized to an individual. By changing colorations ormaking mobile avatars, where the avatars themselves actually move for afraction of a second or multiple seconds, a password/passcode iscreated. One can create an avatar and then connect the same avatar to asection of the face, body part, animal, etc. for encryption thatincludes multifaceted visualization.

Systems for Human Pose Estimation embodying aspects of the presentdisclosure provide the ability to view multiple body partssimultaneously and to store data for comparison at a later date. Forexample, a worker can be recorded in a specific series of motionpatterns for body activity at the start of employment for comparison tolater when the worker complains of an injury at work. Video data may beused to detect a preexisting but unreported injury. The algorithmcompares new video to the older video to identify a preexisting problemto protect the employer and insurer. Similarly, aspects of the presentdisclosure can be used to identify gait and other changes possiblyresulting from a neurological change to detect when an athlete suffers apossible concussion during a game, detect damage from a stroke, assessrehabilitation effectiveness following an injury, monitor improvementsand recovery following a medical procedure or treatment (especially forremote medical care), and the like.

Insurance carriers are able to utilize HPE to compare and alsounderstand if a patient is complying with care and/or if further fundingfor care is needed. Storing video data also includes facial activity,such as twitches and eye movements, that can be keys to pain management,response to medication, symptom magnification, and/or the need formedicines. This is especially important for rehabilitation with opioidsor other pain medications and dealing with stress or anxiety. By storingvideo data and comparing to a baseline-created library that follows thepatient, improved care is possible.

Avatar Human Pose Estimation (HPE): In another embodiment,avatar/characters are created by using Human Pose Estimation. By takingstill pictures or videos of an individual and using Human PoseEstimation type concepts, an emoji-type figure or avatar can begenerated. The avatar can be a static picture or can be a motion pictureof estimation of an individual's face, body, arm, etc. One can also usea motion avatar. In other words, the avatar may have a single plane,multiplane, 3D plane. There can be moving patterns. The avatar can bebuilt off the camera on the user's phone by winking, moving lips orface, or the like. An avatar can be used then to create an emoji basedon the avatar, but this is a motion symbol. One converts this to aletter or a number essentially or create a code. This can be used inreplacement of an alphabet so each avatar or emoji can be a letter ofthe alphabet or a number specifically. This can be used to substitutefor an alphabet and this can be used as a code. It can change on a dailyor weekly basis.

Aspects of the present disclosure permit recognizing and creating anavatar pattern that moves in a single plane, two planes, or in threeplanes (3D) by controlling rotation and linking the HPE to other datapatterns. Simple cameras that can be used on a mobile device or knownsystems for chemical patterns recognition or multiple factors either inrandom or unspecified patterns can be used to drive the transmission ofdata encryption. This can link with the colors, motion patterns, etc. tocreate encryption or data transmission issue. This can also be used withelectrical patterns to move hair follicles or magnet patterns to befixed or utilizing motion, color, or background to link this. This canbe three-dimensional statics or three-dimensional video to create thelanguage for encryption based approaches. This can also be done for ananimal or other moving part. It can be done externally for anenvironment to create a customized avatar figure that can then be usedfor encryption or other technologies.

One can create avatars off cameras, mobile devices, etc. These can besingle 2D pictures, 3D pictures, or avatars through motion. Rather thanmerely using an avatar to describe an activity, one can use the avataras a way to communicate or a way to dispense education/knowledge or tochange the way language is performed or utilized by creating figuresrather than alphabet. They can be static or motion-based avatars thatone can use out of their own body or someone else's to create a languageor barrier by creating certain figures that mean certain words oractivities. These can be standardized despite changes in traditionallanguages are alphabets. The avatars can be simply done with anindividual talking, speaking, or moving. The avatar is based on theuser's particular activity either scanned or changed into an avatar andlinked to a communication system or language system to help individualscommunicate to encrypt data, store data, etc.

DNA/RNA: DNA/RNA encryption can also be utilized in the encryption(i.e., a DNA pattern can be scanned). The DNA/RNA patterns areespecially compatible with computer/mobile devices. This can be throughthe mobile device or attachment to the mobile device where someone mightinsert some piece of body tissue, fluid, blood, etc. One can use Crispertechnology in the body to infuse, change, or assess diseasetransmissions. Crisper technology can be based on mobile devices orcomputer based systems to assess not just encryption or language butalso to be able to assess specific patterns on the surface or within thebody. This can be used for therapy or diagnostics for cancer or abnormaltissue. It can be used for language or features, as well.

Optical Computer Tomography or Optical Coherence Tomography (OCT): OCTis used currently in optometrist and ophthalmologist's office to enhancevisualization of the retina/eyes for assessments. According to anembodiment, a portable OCT-based system, either in a backpack or evensmaller (such as in a mobile device or wearable) can scan not only finesurfaces but also provide a 3D scan or 3D video. Nokia Bell Labs has aportable, battery-operated OCT system for pathogenesis and monitoring ofdisease progression suitable for generating 3D images for use inaccordance with aspects of the present disclosure. This allows one tolook very close at functional activity, surfaces, depth intotissue/around tissue using OCT technology. It may be another that can bebuilt into a wearable or mobile phone concept or can be used fordiagnosis, treatment, and therapy, especially relating toencryption-based technology. These can be implemented in a mobiledevice. It can be in a small separate device that can enhanceencryptions by looking at surface and depth of tissue even going smallnot quite to molecular level but getting down to smaller surfaces in 2Dand 3D and being able to assess any changes or variations. This OCTtechnology with cameras and scanning can be used to look especially atdepth into either encryption but also into education, stress, functionalrecovery, etc.

Environmental Conditions: Environmental conditions such as lookingoutside, looking at the wind pattern against trees, water motion, grasspatterns, etc., can also be utilized for encryption through choosingwhich body part or exercise the system wants to check. One can look atdirt patterns, stain patterns on a wall or window, or complexenvironmental issues that include motion patterns of animals, (i.e.,moving around or running) versus a specific background. These can beused sequentially with known standard encryption systems such as facialrecognition or other factors to determine if still pictures, motionpictures, change in light patterns, etc., are used in an encryption keyor code with a known factor or unknown factor. For example, lava lampsin the past have been used to create random number generators but theirpatterns have been difficult for the user to encode. One can use windpattern, for example, looking at tree motion, water motion, or wavepatterns to create a random number generator, as well. It can beanything that one can look at through video or through multiple stillpictures and place them together. This can be used as these type ofrandom number generators, but it also can be used to generate privacy orlocking mechanisms on a vehicle, phone, computer, etc. for protection.

Eventually mobile devices including potential for OCT can scan at acellular level, molecular level, and potentially even DNA at some point.This allows one a new way to look at encryption but also to look atfunction, recovery, and assess at the same time with wearables and/orchemical sensors, body fluid, hair, functional activities, etc. Thesetechnologies can assess and assist with any type of therapeutictreatment. For example, like CRISPR so one is able to look at technologypatterns even deep inside the human body or at a more molecular levelfor many of these conditions.

This technology is an improvement over the previous technology that areused in facial recognition because traditional solutions are focused ontopographical features and not fine granular features. The currentencryption is not based on specific structures such as specific patternswithin the skin itself, hair patterns, vascular patterns, creases,colors, etc. These can also be linked again to other environmentalconditions, DNA encryption conditions and others. These can bepredictable to unlock phones, look at aging patterns, start self-drivingcars, and for encryption of data (public/private key). In one embodimentthe determination of anatomical features can be implemented through amachine vision and artificial intelligence. In the preferred embodiment,the determination of the facial features, subcutaneous facial features,or any biological feature can be determined with a neural network. Theneural network can be trained with both the user data and a labeled ornon-labeled data set of features that are used for encryption or access.For unlocking the encryption data or device, the compared input data iscompare with the trained CNN and a confidence factor is returnedindicating the likeliness of this matching the biological feature. Thisconfidence level can then be compared to a threshold to determine if thecaptured image should allow access to the protected data. The thresholdcan also be determined by a convolutional neural network which can beused to compensate for changes in appearance such as growing a beard. Inone case the threshold of confidence can be separated into a portion ofthe face without a change and a portion with change (i.e., a beard). Thetwo results can be weighed or used separately. In another embodiment,instead of compensating for the change by adjusting the confidence levelor sample size of the face, the convolutional neural network can betrained such that these changes can be included in the confidence level.In another embodiment, the confidence level required to unlock ordecrypt can decrease with time. This time can be based on a time periodor on a biological parameter.

It is also considered that the facial recognition and human poseestimation can be used to create more complex passwords than justrecognizing the person. There have been known cases where facialrecognition passwords have been compromised by using masks, pictures, oreven unlocking a sleeping person's phone by pointing the phone at them.To create a more secure system, multi-level recognition can be requiredwere the user is able to use a predetermined amount of poses or set acustom amount. To successfully unlock the file, device, or authenticatethe person is required to repeat these poses in order. For example, theuser might set their password to smile, wink, wave, frown. The poses canbe separated by a certain amount of time, where all must be done in atimed sequence or each pose can be evaluated in a state machine.

Because computers are essentially binary, zeros and one, one can createnovel forms of data compression to reduce the data and then one can addthese and code these into digitized avatars that can be used as newmechanisms to transmit and store encrypted data. These avatars can bebased on individual motion patterns or on video clips, for example. Itcan be two-dimensional or three-dimensional. There are companies such asRADiCAL that can assess range of motion in 4D, including time andacceleration, and then allow these to be projected and/or stored onavatar for specific motion patterns or activity patterns. These can thenbe used for data encryption and data storage, but also can be used innumerous medical applications for documentation. For example, surgeryand assistance in helping people recover as well as rehabilitation. Thiscan be used also for medical documentation. It can be HIPAA protected asthey are stored to the Cloud. These can also be used as specificexamples to allow one to create a baseline. If there are any deviationsor changes, one can identify defects. For example, an athlete who mayhave a concussion, one can follow them through video clips and audioclips. If there is a change where one loses memory or activity changessuch as stumbling or falling, one can compare these activities and thendetermine if one in fact has had a concussion or not. These can be usedfor neurological evaluations. For example, during or after a stroke orif someone has had a specific disease or event and how they recoverafterwards. These can be logged to create a baseline in the Cloud andthen look at these functional activities later to assess musculoskeletalconditions, neurologic conditions, functional recovery, sports,activities, etc. to monitor improvements or delays in activities and/orassess mechanisms for improvement and where those are. One can identifymultiple joints throughout the entire body, including facial issueswhere someone can see grimacing, pain, stress, etc. through the eyes orface and then also motion patterns to see how simple, easy, and fastusing acceleration to assess these. These can be added on as an additionto traditional Human-Pose Estimation looking more at 4-dimensionalapproaches and further. These can be used off simple mobile devices.Sensors or wearables can be added to enhance these to look at otherfeatures including biological features, biometric features, etc.

Referring now to FIG. 8 , in another embodiment, aspects of the presentdisclosure monitor additional other changes for irregularities forsecurity reasons. For instance, a credit card size device 802 can becreated with antennas located to interact with credit card stripereaders. These sensors are used to detect the presence of a secondmagnetic coil for reading the credit card strip by looking at theexpected eddy current and changes in the magnetic field when compared toone magnetic strip reader. Secondary readers are hidden in legitimatecredit card readers to “skim” the financial information of anunsuspecting user. One embodiment of this device can have a simple LEDinterface to display if the reader was safe to use or if there was acredit card skimmer present. In another embodiment, as illustrated inFIG. 8 , the credit card device 802 contains a Bluetooth or BluetoothLow Energy radio for interfacing to a smart phone 804. The phoneinterface can be a simple visual indicator of the giving a binary(safe/unsafe) indicator for the credit card reader or, alternatively, aconfidence level of security can be displayed. In another embodiment,the information about the eddy currents and magnetic fields detected canbe sent to the phone 804 for further processing. The phone 804 processesthis data to determine the security level. It is also considered thatthis data can be uploaded to a central repository or the Cloud,indicated generally at 806, to create a larger dataset that can belabeled as suspected skimmer or an unlabeled set.

Using known techniques in artificial intelligence, such a convolutionalneural networks, the uploaded data can then be used to train an AI formore robust detection, returning a confidence value corresponding to thesecurity of the credit card reader. It is also considered the GPSlocation of the scanned credit card reader can be uploaded and sharedwith other users. With this centralized database, the system can sendpush notification to other users who are within a predeterminedgeographical distance to credit card readers suspected of skimming.Alternatively, the application can regularly download the entire datasetor a subset of the data of known skimmer locations based on currentgeographic locations, and a geofence can be created around suspectedskimmer locations. If the user crosses the geofence, a notification(alert, popup, vibration, sound) can be sent to users to alert themabout the suspected skimmer. It is also considered that based onprevious detection of skimmers at a location, or ones in closeproximity, that there is a predicted risk at a location instead of onlyreporting detected skimmers. Alternative embodiments can use WiFi, RFID,ZigBee, or other wireless protocols for wireless data transfer. If askimmer is detected at a location, the software can allow the businessuser to have the alert removed from the database or marked resolvedafter it was removed or investigated. In one embodiment, the applicationcan give the option of flagging or canceling a credit card that issuspected of being skimmed.

Another embodiment is a scam alert system (app, website, identity,banking, etc.) that alerts (a warning/alert) both consumers andcompanies about scams. This can also include other types of scamsincluding ticket scams, phone scams, etc. This can even includeinforming law enforcement to further investigate. This can also includemap of where scams are most prevalent and a rating scale of thelikelihood of it being a scam. There is also a rating system for low(possible) medium (multiple reports) or high (proven)scams, locations orbusiness/individuals. This can be packaged onto a website orApplications that automatically alerts users to a number, location orbusiness that contacts individual or that individual may contact andrate the risk.

Referring further to avatars and/or motion avatars, aspects of thepresent disclosure contemplate their use for language encryption andeven blockchain-based technologies. Use of a motion or character thathas some limited movement can enhance and shorten communication, createa more versatile and universal language free from static language bias(e.g., bark can be either a dog sound or covering of a tree) that mustbe interpreted in context of sentence, paragraph. Motioncharacters—avatar being one type—may enhance communication andspecificity despite language or dialect or regional differences.Language is based on static characters or symbols (i.e., ABC), which arestatic and limited. There can be an emoticon dictionary containing oneor a combination of avatars to help create a universal language andclarify what the avatars mean for all languages. This can also includecultural differentiation of the meanings of avatars. One example of thisa common gesture when a person from South Asian cultures where a tiltingthe head from side to side can mean “yes” or “good”. In other cultures,this nonverbal communication is done by moving the head up and down. Inone embodiment, animated avatars can automatically update to reflectcultural shifts in the meaning of the motion or emotion being conveyedbased on geographic location, keyboard language, or context. Forexample, FIG. 9 illustrates two emoji-type avatars, one that nods up anddown and the other that rotates side to side. These avatars can havedifferent colors. One can add certain symbols, numbers, or letters. Itcan be used to create an international language, for example, usingavatars, either single avatar or motion based avatars. Avatars can beused with other figures or symbols next to them so one can create alanguage for communication, technology, encryption, trading,transmitting data, or sharing of information. This can reachinternational lines as avatars are transmitted in a binary code throughthe computer. However, they allow individuals to communicate withdifferent languages, different cultures, and different alphabets. Thismay be a way to create a language and can be added on to quickly so thatusers of avatar language can become fluent easily. Varying of the color,shape, or background pattern can be implemented as well. For example,polka dots versus flat background versus different color schemes. Onecan use a three-dimensional avatar single plane, multiplane, motionplane, rotational plane.

In an aspect, the use of avatars can remedy the discrepancy of languagewhere one word may have multiple meanings in different languages. Onecan also use avatars where motion patterns can be then downloaded viamobile devices for example and whether these are simple facial, entirebody, or activity. It can be done either through motion or staticfigures. It can be logged on as an avatar and then create language orencryption based technology on these motion devices. They can be otheruniversal characters so that specific languages and/or alphabets can bebypassed for a more international discourse. In the past, all staticsymbols have been used for different alphabets whether Chinesecharacters or Roman alphabet figures. Here, static images and/or motionimages singularly or in combination create nouns, verbs, and adjectivesthrough avatars (moving or static) or in combination with traditionalalphabets to try to make more universal language or communication.

Avatars can also be used for distributing/computing via blockchain usingnew language keys. They can be used in single or multiple computers.They can be used for cryptocurrency, NFT, or as a way to assess orenhance blockchain rather than using standard alphabet or characters,which have significant limitations.

Aspects of the present disclosure also can combine an emoji from acamera (such as shown in FIG. 7 ) where one creates language patterns byscanning facial, body, or motion features linking these to some type ofemoji or video feature and then these can be encrypted and moved. Tocreate a more descriptive emoji the person can have his avatar addedand/or emojis added on top of other emojis, resulting in a set of actionemojis to best describe what the person intended. Rather than just anemoji, it can be the user's own personal avatar that comprised of, forexample, a scan of the user's own body downloaded into specific featuresin binary 1's and 0's. They can be amplified. It can match what yourbody is and this can be used for language, data transmission,encryption, or protection. These can be used also to integrateencryption technologies for communication. This language can be used formedical information transfer, technology, and other applications.

An avatar and/or audio can be used for this language communication. Onecan use these for financial data, for assessing problems, to controldata, and to add other features for smart phone encryption. One can usethese for example in DocuSign. Rather than a simple signature, which is2-dimensional and can easily be copied or forged, one can use athree-dimensional system and adding many of these other encryption-typefeatures to enhance DocuSign. This can also be used on a check. Forexample, if one wants to communicate this or for a credit card ratherthan a simple chip, one can also link the chip to your mobile phonewhere the mobile phone takes a video of you and/or your specific motionpatterns and/or section of skin, facial recognition, multiple aspects,etc. that can ensure one was using the credit card, one was writing thecheck, or one doing the DocuSign actually is the individual doing it.This is proof either at the time through a delayed management where onecan double check this through a Cloud-based approach or it is stored forproof later for legal issues, government issues, police issues, and thelike to prove or disprove that one in fact signed the document, agreedwith a certain activity, or legally accepted a certain contract. Thiscan also be further used for verbal contracts or verbal communicationwhere someone says “I agree” to this. One can also scan the individualbody as well looking at the face, eyes, or a multiplicity of theseencryption-based approaches. For example, did that individual in factapprove of this? The user can then store and download it into the Cloud.One can prove that individual by certain motion patterns, chemicalpatterns, functional patterns, or body patterns and not just acousticsound, simple video, or simple signature. One can prove individual basedon further encryption technologies that proves this was the individualand that they did approve a certain activity, function, financialtransaction, etc. This is also a novel way to look at credit cards,credit card authorizations, check authorizations, and make this morepersonal going back to the first paragraphs and discussions onencryption based approaches.

Credit card theft is massive. If the credit card links to your mobiledevice, the mobile device then has a particular pattern as well as hasaudio/video approvals and immediate understanding. One can double checkthose patterns with the individual himself/herself did approve thistransaction and this is proof. Credit card companies can see this overthe Cloud rather than simple text message that confirms this or does notconfirm this. It is an actual picture with e-mailed video of theseencryption type technologies and can be used to confirm the individualdid approve it and/or later for credit card companies to send billingand confirm this is the individual that did this and they do requirethese type of payments. This can be used for legal and/or financialtransactions and/or proof of these.

This can be evaluated and stored on a phone or compact flash. It can bestored in a computer. It can be carried with you and information can bestored to flash memory rather than to the Cloud for encryption orsecurity.

In another aspect, Human Pose Estimation links to an emoji or an avatarthat one can use for encryption. People have specific motion patternsthat are not just their movement, but also to acceleration and rotation.As users change or age, their patterns of activity (whether in upper orlower extremities), motion patterns, facial tics, etc., can then bedownloaded to an avatar or emoji to create language issues. It can alsobe used for privacy or encryption issues or for data transmission. Thesecan be varied on a regular basis to help even with privacy issues, sothat companies are not mining all your data from your e-mails or acellphone. When utilizing this, one can modify activities to confuse,preserve data, and prevent social media from mining, thereby creating amoving target. One can use these same types of concepts as they relateto environmental factors, personal factors, motion factors, or linkingHPE and others to present multiple addresses (either single source ormultiple sources) to block data mining and/or create facets for privacy.Again, this can also be used for wealth preservation and/or financialissues through blockchain or other financial institutions that allowsmobile funding. In addition to using this information as a key orpassword, the generated codes in the disclosure can be used with knownVPN or anonymous browsing tools as TOR to anonymize browsing and protectpersonal data, and can also be used for Blockchain.

This can be used to prevent browser theft of data creating mobileaddresses from spreading out to other addresses. This can also be anend-to-end encryption technology or can link random codes to fix codes.This can be audio or video or linking both. It can be linked to anavatar which can be walking, gait, bouncing, rattling sound, motion,shaking, etc. As described above, a smart phone or wearable including anaccelerometer provides acceleration data as the user moves in the x, y,and z directions. For example, a cellphone in your pocket as you movearound may create a number of different issues as it relates to motionpicked up by accelerometer or gyroscope. It also creates a sound as itrubs against certain materials and/or video. It may be able to createthese motion patterns with or without Human Pose Estimation or avatarsto create data links, encryptions, privacy features, mobile addresses,etc. These concepts are linked with additional data to either trainneural networks to create encryption values, to create lockingmechanisms, to create end-to-end encryption technology, to createvarying addresses, to create a new language, or for distributing orcomputing. It can be used for mobile financial networks as examples thatrequire huge data power, as well as a lock or key for encryption orencryption-based approaches.

It is also considered that as a lexicon of emojis are built that itmight not require a centralized database to host the images ordictionary for these emoji. By leveraging blockchain, this informationabout usage, definition, or graphical rendering of new icons can beupdated through blockchain's distributed ledger. The computationalburden by adding blockchain can be performed by multiple methods. In oneembodiment, the user can run some computations while the incoming datais being loaded and/or displayed. In other cases, the computations canbe done offsite. These computations can be supported by ad revenue in anapplication or fee for use. These computations can also be used tocreate the electrical energy required to mine cryptocurrency or utilizeblockchain or other types of technologies. U.S. Patent ApplicationPublication No. 2018/0355837, the entire disclosure of which isincorporated herein by reference, discloses capturing energy fromnatural resources, such as movement of fluid in a body of water, andconverting it into electrical energy. The language developed in theseprocesses can be used in advertisements, arts, NTFs, or otherapplications. These can then be used in finances, recognition, ordifferent types of communication that protect individuals or directspecific individuals.

Rather than audio to create language or communication which is based onmultiple alphabets languages and dialects, here a language is created byPhone/Mobile Video systems that monitor movements and convert movements,activity, color, or motion to a “video language”. Thus, it isinterpreted by machine learning or AI algorithms on receiving side anddirects the user as to how to create or educate on “language”. A simplerand faster way to communicate is thus created, especially since smarttechnology so ubiquitous. One can also add audio, emoji, or multiplefeatures rather than linking to a single feature. On the language sidethis can also be a new way to program computers or transmit informationrather than the typical 0 and 1 binary code, where instead computersthat can communicate with more than 2 symbols—for example a clock facewith multiple positions and or the motion or color or 3D interpretationHPE—has led to the concept that computers can recognize motion ratherthe traditional on/off 0/1binary code, which has limits. Also, theprocessor is changed to accommodate multiple layers, like a 3Dchessboard or tic-tac-toe in 3D, and a user can connect in differentdirections or angles.

Referring now to FIG. 10 , aspects of the present disclosure utilizemixed reality or virtual reality goggles 1002. In the illustratedembodiment, the goggles 1002 includes an internal camera array 1004where one has cameras that face inward to look at the eyes, face, stressissues in/around the eyes, pupil dilatation, sweat, pulse, heart rate,etc. and other cameras that look externally to the particularenvironment to enhance the environment while one is using augmentedreality, virtual reality, or mixed reality. In an aspect, the goggles1002 include a display that fully covers the wearer's eyes to provide animage to the wearer in his or her central and peripheral vision. Thecamera array 1004 of the goggles 1002 provides visual data to acontroller coupled to the goggles 1002.

In one embodiment, the controller is internal to goggles 1002. One canhave one or multiple cameras either fixed to goggles 1002, fixed to theindividual, or fixed to the room where one is working out of. Thecameras external to goggles 1002 can be mobile device cameras. Forexample, two wrist watches with cameras can look at how an arm or legmay function in relationship to a certain procedure or certain activityand then can stimulate to enhance the arm and/or leg either throughindividual or through neurologic stimulation, electrical stimulation, orthe like to change the motion pattern to enhance functional recoveryand/or to improve function, speed, efficiency, reduce calories, etc. tomake an operation faster whether it is in the surgical suite or whetherit is in a work related environment to improve efficiencies and results.These can also be used for exoskeleton to make an operating system work.In an embodiment, the exoskeleton can be linked to sensors on atraditional body whether it is in the clothing, shoes, gloves, etc.where one can operate an exoskeleton robot or sync onto otherindividuals in an environment to enhance their functions or work-relatedcapacities or potentially to link to robotic systems desired to functiontogether. Robotic systems may be similar hydraulic, servomotor, etc. orthey can be different types such as one might be ingestible, one mightbe external, or one might be servomotor. It allows one to operatemultiple robotic systems and mechanical symptoms in synchrony usingthese type of concepts with wearables that are embedded in clothing,gloves, etc. linking these to video patterns, and then allowing one tolink camera, sensors, etc. using either AR or VR and also audio to givethese verbal cues. The verbal cues, again, can be used with new languagebased technologies with avatars, audio, etc. to allow more rapid motionpatterns, more rapid functional patterns, rehabilitation, recovery, orimprovement of functional activities for athletes, etc.

MirrorAR has two-dimensional imaging using mobile camera/mobile phone.RADiCAL uses four-dimensional, which includes acceleration and looks atrotational motion. These are predicates but can be used in addition toassess rehabilitation, recovery, neurologic injuries, concussions, orwork-related issues by having a baseline and assessing baseline ofmultiple joints, including upper/lower extremities and spine insynchrony and then compare these to video a week, a month, and a yearlater. This looks for any changes, improvements, or deterioration infunction. These can be used for athletic performance, musculoskeletalperformance, and recovery from surgery. These can also be used forgeneral rehabilitation and/or more importantly for medical records whereone can diagnose more efficiently on a medical record. For example, oneassesses a knee and subjectively one describes 10 to 90 degrees arc ofmotion. However, with this video-based program using a mobile deviceshooting at any angle and any direction without any external wearablesone specifically sees range of motion not just of the knee, which can bedocumented, but in the same assessment one can also see the hip, spine,shoulder, upper extremities, etc. One can later follow if someone has ahip-based limp. One can also see diagnostics for the foot and ankle orother issues. This can be used for medical diagnosis not only but alsofor medical documentation to improve medical records. Companies such asCerner or Epic use secondary data that is vocally recorded and is tryingto use words to describe complex musculoskeletal motion patterns. Thischanges the whole concept, so one does see not a single joint butmultiple joints. One can compare them with current mobile devices, sonot just for remote medicine but also for musculoskeletal and neurologictreatments. In the office, one has this type of data. This can be usedin surgery to determine what the diagnosis was but also what was doneduring surgery. One can link the audio to the video component so thatone can actually hear opinions and statements as well as look at thebody motion and then compare it a week, a month, and a year later usingthese types of algorithms to diagnose, treat, assess improvement/decay,and then warn individuals whether additional therapies or treatments arerequired.

As an example, if one does not improve quickly enough, does he or sheneed additional therapy and what type of therapy as well as what shouldone look for. This also aids a physical therapist to see where thepatient's overall body function was. Again, with a simple video, onedoes not just look at the knee if the examination is based on the knee,but one can now look at the hips, spine, or other joints through motion.These can be proscribed motion. For example, one can do sitting tostanding, kneeling, squatting, or jumping jacks looking at arms overheador hands moving looking at multiple complex joints of the hands andwrists by setting up specific motion patterns. It may take one, two, orthree seconds. They are stored in the Cloud and then compared to thesame motion patterns on the next patient visit or from a home-basedapproach. These are then compared to see if there is any change. Thesecan be used for diagnosis, treatment, or enhancingrecovery/rehabilitation. It is also very useful for athletes. One canlook for concussions, injuries, or recovery. It can help determine whenone is able to return to work. One can use this for Worker'sCompensation challenges. For example, it identifies if someone says I donot have back problems or hip problems, but take a video as the initialassessment of someone at work. One can then see if there is somelimitation in function or motion. If someone comes back six months laterand says my knee was injured at work; but, if one has documentation andthere was decreased range of motion at the knee, limp at the knee,spine, or back, one can say I have definitive proof that one in fact didhave limitations and work may not have directly caused the injury orthere were pre-existing conditions affecting overall function. This canalso be used to screen out patients, for example, who may have high riskfor back problems or knee problems to avoid certain occupations. It canbe used as a standard for assessing this and a risk management program.Human-Pose Estimation essentially takes video clips using mobile devicesand then stores them to the Cloud. During this, they are quicklyassessed for rotational moment around certain joints.

These can be then used in addition with wearables that one can wear onthe arm or leg that may give more specific function about a specificjoint and link these to the video or they can utilize general parameterssuch as look at a pulse oximeter or an Apple watch for example thatmight test EKG, pulses, blood pressure, etc. These are linked to thevideo assessment to then give the overall assessment. These can be donefor remote patient care, but also to have a baseline. These can bestored in an individual's phone and computer or stored to the Cloud orat the doctor's office. One can compare one video to the next. With AIalgorithms, even if one is 20, 30, or 40 degrees off with a camera ordistance may be a foot or ten feet, one can assess these and then beable to compensate with AI algorithms to assess the distance and stilllook at rotational arc of motion and especially the acceleration, howquickly one moves as this also gives an insight into the patient'srecovery or activity.

The same can be used for facial recognition. For example, at the sametime one is looking at the arm or leg, one can see the stress on anindividual's face and how hard one has to work. If one is in substantialpain, it can look at contraction around the eyes, sweating, stress,pupillary constriction dilatation. One can also combine these withwearables to look at sweating or other issues that may assess stress,pain, and function. These can also be used for testing for specificdrugs or pharmaceutical success and/or efficacy. These can be usedwhether a pharmaceutical company is testing or whether physician is notfinding the right medication. It helps determine whether the medicationshould be changed or if it is at all effective or not for thatparticular individual. This also links for example to other issues suchas food intake, exercise, distance, movement patterns, time of day, ornighttime to link to success efficacy of medicine, dosage, and/ortreatment.

Again, these Human-Pose Estimation concepts are looking at video motionassessments and can easily be done allowing remote patient care andmonitoring, but further for documentation of the patient's medicalconditions, recovery, diagnosis, treatment, therapy, etc., especiallyfor musculoskeletal, neurological, etc.

Yet another use of this technology is as a replacement for a signature.One signs a letter and one's signature can be easily modified or copiedby someone else. These can be used in specific patterns such as DocuSignor others. Rather than using DocuSign, we can create an entire systemwhere one uses someone's body encryption or use a portion of a body,DNA, motion patterns to create a new or novel system. Rather thansigning a document, one can scan and then place one's own DNA, skinpatterns, hair patterns, etc. to create signature for signing a documentfor real estate, someone's will, moving finances or money. Again, thisrelates to blockchain where if someone wants to transfer different typesof funding one can use these technologies to encrypt funding or todirect funding into specific technologies or locations, moving realestate, moving a will, or moving finances from one bank to another sobanks have a novel way to encrypt or move technologies. For example, itcan be from the Fed to a bank so it would not be copied or stolen usingsome of these technologies. These can also be used to translate tolanguages or create your own language so one can give confidentialinformation from one source to another. For example, the 26 characterswe have in the English language or the different characters or figuresthat are used in Asian languages, hieroglyphics, etc., one can create anemoji or some type of matched pattern where one can use a camerareversed upon themselves to do motion, discussion, or speech and theneither audio, video, etc., to create new patterns to allow datacommunication which can then be back dropped and encrypted into a newpattern for a more universal language or for encryption based onletters, characters, sentences, action. These can be used for a singlefigure, single letter, sentence, paragraph, or entire motion platformcan be used.

Another cause of changes in body composition is stress. In anembodiment, body tissue is stressed to look for specific tissue whetherit is from medical diagnosis and encryption as these keys discussedabove or whether it is looking at controls. One may have a reasonablevascular pattern. However, stresses such as static electricity,warm/cold air, fluid, and stimulants may change the way hair folliclesor creases occur or cause other bodily changes. Temperature stresses,for example, cause pupils to dilate, sweating, etc. For example, if onewets the skin, it may look different and this can be another way to“encrypt or create security” so it is not simply standard. Adding heatto warm the skin causes the skin and blood vessels to dilate resultingin a vascular pattern change. Therefore, one may be able to use a stressconcept in addition to standard to create further level of securityencryption or control. One can look at this in a cubic centimeter areaas an example and look at other ways to stress the tissue through otherforms other than electrical, thermal, compression, external compression,suction, vacuum, etc.

The use of an AI system that combines the use of a visual and audioexamples to detect if a person is lying, stressed, tired, etc. or not(can be used with a VR headset, such as shown in FIG. 10 ). One suchexample of a person most likely telling a lie is that their pupilsdilate. This can also possibly detect concussions or other injuries(such as a concussion or a ACL tear) through a comparison to a baselinetest.

In addition to determining unique bodily features for generatingencryption keys to secure data and the like, aspects of the presentdisclosure are well-suited for discerning lies from truth. One cannotsimply look at stressing the individual through questions, such asduring a lie detector test, which essentially looks for electricalchanges. Using video to look at musculature tick or motion patterns, forexample, can reveal a “tell” that an individual has where one differsfrom another. As an example, when individuals play poker, they candetect if someone is cheating, lying, or telling the truth. One can usevideo or Human-Pose Estimation, but also looking at this from othermechanisms to test whether someone is accurate or inaccurate. This canbe used for medical applications such as Worker's Compensation. It canbe used for legal applications. It can also be used for otherapplications for even encryption. One can look at a standard video of anindividual and then stressing individual whether through verbal cuessuch as questions, thermal, electrical, visual, etc. and see how theindividual responds and using these possibly in combination withquestioning to determine whether someone is telling the truth accuratelyor not. These can be started by looking at one's baseline activity andthen using these Human-Pose Estimation or encrypted videos of thepatient's face or body with not just single stress of questions butmultiple stresses. These do not have to be severe. They can besuperficial such static electricity or mild thermal changes to the room.It can be used by for example changing oxygen content. As the oxygenincreases, one can change the stress or increase the carbon dioxide andthen apply stress to an individual and then question. One can determineand/or compare if there is a change.

One can assess also legal accuracy in new standard rather than lookingat simply a lie detector, which used electrical signals. One can thenvideo and look very close at the eyes and assess for this. One can alsostrap-on through wearables, chemical sensors, sweat. The idea of hairfollicles moving the affected vessels dilating or contracting which canuse infrared as well as looking at dryness of mouth, facial tics, or“tell” as poker players call it where one can see if one has a change orrepeated pattern. Again, this can be done through video assessment,i.e., Human-Pose Estimation where one can look at the body, scan it, andthen have a baseline. If one identifies any changes with this, one isable to see if there is accuracy/inaccuracy of data or if someone'smental status, pain, etc. matches what they are actually broadcasting,saying, or doing.

Examining body segments for reactions to stress can also be used todetermine diagnosis/activity, for example concussions or neurologicinjuries. It can also be used to, for example, predict impendingcardiac, neurologic, or other stress issues to the body. For example,diabetic stress testing has a patient ingest a huge amount of sugar in aliquid and then measure insulin levels and glucose levels. As the bodyresponds and if the body responds abnormally, one can determine whetherone is a pre-diabetic or at risk for diabetes. These same types ofconcepts can be used to follow individuals for accuracy or risks forother diseases. For example, if one stresses them, do they haveappropriate response for hypertension or appropriate response forcardiac issues. Currently, one does a stress test for the heart whereone can be put on a treadmill, elevate the treadmill gradually andprogressively to see how their blood pressure responds. One can also addother variables as we discussed by changing the oxygen, changing thecarbon dioxide, and changing the temperature. By following the response,one may be able to determine more granularly what type of cardiacdisease this is specifically, whether it may be a metabolic disease suchas thyroid versus cardiac, whether one may be secreting excessiveamounts of epinephrine, or if one may have kidney or essentialhypertension issues rather than cardiac issues as simple ways to enhancediagnostic capabilities. Again, looking at these, controlling thevideos, measuring pre and post or during these activities and thenadding stress components, this may enhance one's ability to diagnose notsimply just for encryption, although these can all be part of the samechallenges as we are aware certain people respond very poorly to stressor anxiety and they need anxiolytic medications.

Applying stress to individuals can be measured on different types ofwearable devices, which are commonly available. If they are stressedappropriately, one can determine and place these in a program as theystress for specific parameters and measure this. They then look and seehow the patient responds. For example, if we increase the oxygen in aroom or if we stress them by increasing the carbon dioxide or increasethe temperature two to four degrees either locally in the room or in theregion or for example immersing a body part such as a foot in warmerfluid or having someone exercise, then one can see if their body changesin terms of their response. One can predict diseases by stressing thebody and disease management. These can also be used for these securityor encryption based technologies and can be measured easily by manywearable devices. These can be used for remote medicine. These can bedone at home by individual or by not necessarily medical professionalsbut through remote patient care and monitored to assess. For example, ifsomeone has asthma, how can one assess and control it. What is thestimulant for this? If one can stress in a local environment to followthis, one can be able to measure this. For example, mobile devices maybe able to measure oxygen and carbon dioxide through inhaled or exhaledair. If someone has significant allergies to hay, for example, how theyshould respond and what medications should be taken may be determinedbased on the stress reaction caused by exposure to hay. One theymedication is taken, how do they respond and if these parametersimproved. These can then be sent in via the cloud or to healthpractitioner until HIPAA protected mechanisms to determine thetreatments. This can change the way medicine is performed by doingindividual stresses for individual diseases.

In yet another embodiment, adding encryption based on a 3D image,pattern, or motion can be used to replace or add to existing bar codesfor pricing or inventory control or tracking or other scanning codes,keys to security, self-based encryption systems, keys, government issuedIDs, and the like.

Embodiments of the present disclosure may comprise a special purposecomputer including a variety of computer hardware, as described ingreater detail herein.

For purposes of illustration, programs and other executable programcomponents may be shown as discrete blocks. It is recognized, however,that such programs and components reside at various times in differentstorage components of a computing device, and are executed by a dataprocessor(s) of the device.

Although described in connection with an example computing systemenvironment, embodiments of the aspects of the invention are operationalwith other special purpose computing system environments orconfigurations. The computing system environment is not intended tosuggest any limitation as to the scope of use or functionality of anyaspect of the invention. Moreover, the computing system environmentshould not be interpreted as having any dependency or requirementrelating to any one or combination of components illustrated in theexample operating environment. Examples of computing systems,environments, and/or configurations that may be suitable for use withaspects of the invention include, but are not limited to, personalcomputers, server computers, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, mobile telephones, network PCs, minicomputers,mainframe computers, distributed computing environments that include anyof the above systems or devices, and the like.

Embodiments of the aspects of the present disclosure may be described inthe general context of data and/or processor-executable instructions,such as program modules, stored one or more tangible, non-transitorystorage media and executed by one or more processors or other devices.Generally, program modules include, but are not limited to, routines,programs, objects, components, and data structures that performparticular tasks or implement particular abstract data types. Aspects ofthe present disclosure may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotestorage media including memory storage devices.

In operation, processors, computers and/or servers may execute theprocessor-executable instructions (e.g., software, firmware, and/orhardware) such as those illustrated herein to implement aspects of theinvention.

Embodiments may be implemented with processor-executable instructions.The processor-executable instructions may be organized into one or moreprocessor-executable components or modules on a tangible processorreadable storage medium. Also, embodiments may be implemented with anynumber and organization of such components or modules. For example,aspects of the present disclosure are not limited to the specificprocessor-executable instructions or the specific components or modulesillustrated in the figures and described herein. Other embodiments mayinclude different processor-executable instructions or components havingmore or less functionality than illustrated and described herein.

The order of execution or performance of the operations in accordancewith aspects of the present disclosure illustrated and described hereinis not essential, unless otherwise specified. That is, the operationsmay be performed in any order, unless otherwise specified, andembodiments may include additional or fewer operations than thosedisclosed herein. For example, it is contemplated that executing orperforming a particular operation before, contemporaneously with, orafter another operation is within the scope of the invention.

When introducing elements of the invention or embodiments thereof, thearticles “a,” “an,” “the,” and “said” are intended to mean that thereare one or more of the elements. The terms “comprising,” “including,”and “having” are intended to be inclusive and mean that there may beadditional elements other than the listed elements.

Not all of the depicted components illustrated or described may berequired. In addition, some implementations and embodiments may includeadditional components. Variations in the arrangement and type of thecomponents may be made without departing from the spirit or scope of theclaims as set forth herein. Additional, different or fewer componentsmay be provided and components may be combined. Alternatively, or inaddition, a component may be implemented by several components.

The above description illustrates embodiments by way of example and notby way of limitation. This description enables one skilled in the art tomake and use aspects of the invention, and describes severalembodiments, adaptations, variations, alternatives and uses of theaspects of the invention, including what is presently believed to be thebest mode of carrying out the aspects of the invention. Additionally, itis to be understood that the aspects of the invention are not limited inits application to the details of construction and the arrangement ofcomponents set forth in the following description or illustrated in thedrawings. The aspects of the invention are capable of other embodimentsand of being practiced or carried out in various ways. Also, it will beunderstood that the phraseology and terminology used herein is for thepurpose of description and should not be regarded as limiting.

It will be apparent that modifications and variations are possiblewithout departing from the scope of the invention defined in theappended claims. As various changes can be made in the aboveconstructions and methods without departing from the scope of theinvention, it is intended that all matter contained in the abovedescription and shown in the accompanying drawings shall be interpretedas illustrative and not in a limiting sense.

In view of the above, it will be seen that several advantages of theaspects of the invention are achieved and other advantageous resultsattained.

The Abstract and Summary are provided to help the reader quicklyascertain the nature of the technical disclosure. They are submittedwith the understanding that they will not be used to interpret or limitthe scope or meaning of the claims. The Summary is provided to introducea selection of concepts in simplified form that are further described inthe Detailed Description. The Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used as an aid in determining the claimed subject matter.

What is claimed is:
 1. A method for protecting data, the methodcomprising: acquiring initial image data from a user at a first time,wherein the initial image data is representative of a movement patternof the user and represents a unique biometric feature of the user,wherein the initial image data includes gait data representative of themovement pattern of the user, and wherein the unique biometric featureof the user comprises a Human Pose Estimation derived from the gaitdata; generating, based on the initial image data, a key associated withthe unique biometric feature; encrypting a data file using the key;acquiring subsequent image data from the user at a second time laterthan the first time; executing an image engine configured to determinewhether the subsequent image data matches the initial image data,wherein the image engine is trained to create a confidence level formatching the initial image data with the subsequent image data;unlocking the encrypted data file in response to the confidence level ofthe image engine indicating the subsequent image data matches theinitial image data within a predetermined threshold; and generating anavatar associated with the user based on the Human Pose Estimationderived from the gait data, wherein the avatar is a visualrepresentation of the user.
 2. The method as set forth in claim 1,wherein the image engine comprises at least one of the following tooptimize the confidence level: a neural network; machine learning;machine vision; and artificial intelligence.
 3. The method as set forthin claim 1, wherein the initial image data and the subsequent image dataare of the same data type.
 4. The method as set forth in claim 1,wherein the initial image data and the subsequent image data compriseone or more of the following data types: gait data, user skin traitdata, user blood trait data, environmental data, DNA data, user retinaldata, and user fingerprint data.
 5. The method as set forth in claim 4,wherein the user skin trait data comprises one or more of skin creases,wrinkles under the eyes, number of hairs per area, vascular patterns,and skin irregularities.
 6. The method as set forth in claim 1, whereinunlocking the encrypted data file comprises at least one of thefollowing: unlocking a smartphone, unlocking a computing device,decrypting the encrypted data file, unlocking a vehicle, and authorizinga transaction.
 7. The method as set forth in claim 1, further comprisingassociating the avatar with the unique biometric feature, and whereinthe key is generating based on the avatar.
 8. The method as set forth inclaim 1, wherein executing the image engine comprises executingartificial intelligence to predict changes in the unique biometricfeature of the user over time.
 9. A system for protecting datacomprising: a camera configured to acquire initial image data from auser at a first time and subsequent image data from the user at a secondtime later than the first time, wherein the initial image data includesuser skin trait data of the user and represents a unique biometricfeature of the user, wherein the initial image data further includesgait data representative of a movement pattern of the user, and whereinthe unique biometric feature of the user comprises a Human PoseEstimation derived from the gait data; a processor; and a memory storagedevice, wherein the memory storage device stores processor-executableinstructions that, when executed, configure the processor for:generating, based on the initial image data, a key associated with theunique biometric feature; encrypting a data file using the generatedkey; executing an image engine configured to determine whether thesubsequent image data matches the initial image data, wherein the imageengine is trained to create a confidence level for matching the initialimage data with the subsequent image data; unlocking the encrypted datafile in response to the confidence level of the image engine indicatingthe subsequent image data matches the initial image data within apredetermined threshold; and generating an avatar associated with theuser based on the Human Pose Estimation derived from the gait data,wherein the avatar is a visual representation of the user.
 10. Thesystem as set forth in claim 9, wherein the image engine comprises atleast one of the following to optimize the confidence level: a neuralnetwork; machine learning; machine vision; and artificial intelligence.11. The system as set forth in claim 9, wherein the initial image dataand the subsequent image data are of the same data type.
 12. The systemas set forth in claim 9, wherein the initial image data and thesubsequent image data comprise one or more of the following data typesin addition to user skin data: gait data, user blood trait data,environmental data, DNA data, user retinal data, and user fingerprintdata.
 13. The system as set forth in claim 9, wherein the user skintrait data comprises one or more of skin creases, wrinkles under theeyes, number of hairs per area, vascular patterns, and skinirregularities.
 14. The system as set forth in claim 9, whereinunlocking the encrypted data file comprises at least one of thefollowing: unlocking a smartphone, unlocking a computing device,decrypting the encrypted data file, unlocking a vehicle, and authorizinga transaction.
 15. The system as set forth in claim 9, wherein thememory storage device stores processor-executable instructions that,when executed, further configure the processor for associating theavatar with the unique biometric feature, and wherein the key isgenerated based on the avatar.
 16. The system as set forth in claim 9,wherein executing the image engine comprises executing artificialintelligence to predict changes in the unique biometric feature of theuser over time.